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Abstract — Reliability demonstration testing for software 
products is performed for the purpose of examining whether 
the specified reliability is realized in the software after the 
development process is completed. This study proposes a model 
of reliability demonstration testing for continuous-type 
software such as that for controlling production lines and 
operating systems of computers. Testing time and acceptance 
number of software failures are designed based on variation 
distance. This model has less parameters to be prespecified 
than the statistical model. 

Index Terms — software reliability, reliability demonstration 
testing, continuous-type software, variation distance 

I. Introduction 

The studies of software reliability evaluation can be 
classified into those of analyzing data obtained in the 
development process of a software product and those based 
on software reliability demonstration tests after the 
development process. As to the former, there are studies on 
estimating the reliability evaluation measure in the 
development process or estimating development periods 
needed to achieve the intended reliability [1, 2]. On the other 
hand, when the development process is completed, a reliability 
demonstration testing which has been developed for 
hardware products originally [3] should also be applied to 
software for the purpose of examining whether the specified 
reliability has been attained [4]. 

We have suggested models for reliability demonstration 
testing of continuous-type software such as that for 
controlling production lines and operating systems of 
computers [5]. Testing time and acceptance number of 
software failures are designed based on the concept of a 
statistical test, which requires us to specify the values of 
producer's and consumer's risks. This study proposes a model 
of reliability demonstration testing for continuous-type 
software on the basis of variation distance [6], which can be 
regarded as a measure to express the distance between two 
probability distributions. This model can design the test more 
easily than the statistical model since this model includes 
less parameters to be prespecified than the statistical model. 

II. Notation and Assumptions 

The reliability of continuous-type software is discussed 
in terms of MTTSF (Mean Time To Software Failure),MTBSF 
(Mean Time Between Software Failures), SFR (Software 
Failure Rate), and the number of remaining faults in the 
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software. The continuous model for software reliability 
demonstration testing deals with such a type of software. 
Under the continuous models, the software of interest is 
tested for time ?, and is accepted if the number of software 
failures in the test does not exceed an integer s and rejected 
otherwise. In this case, the design variables are t (t > 0) and s 
(5 = 0,1,2,...). 

The notation used in this paper is as follows: 

O MTBSF on the contract 

e i Tolerable lower limit for MTBSF ( d, < 6 ) 

E 1 Event that the number of software failures during the test 

exceeds s 

E l Complement of E { 

The assumptions made throughout this paper are listed 
below: 

(i) No fault is removed during the test. All the faults for 
software failures during the test are removed after the test is 
completed. 

(ii) The software failure times follow an exponential distribution 
with mean# , i.e. the number of software failures on (0, t] 

follows a Poisson distribution with mean t IB . 

(iii) The values of 9 and 9 i are specified at the beginning of 
the test. 

Assumption (ii) is relatively conventional in the software 
reliability models. 

III. Statistical Model 

In this section, we describe a model based on the concept 
of a statistical test with a view to determining the values of t 
and 5 in the above SRDT 

In the above SRDT, the probability that event £ 1 occurs 
when 8 =9 is is given by 



(1) 



Likewise, the probability that event g l occurs when 6=6 
is expressed by 



(2) 
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The probability in Eq. (1) is called a producer's risk in SRDT 
as well as in sampling theory, while that in Eq. (2) is 
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called a consumer's risk. The producer's and consumer's risks, 
respectively, signify the probabilities of Type I and Type II 
errors in terms of a statistical test. 

When the values of the producer's and consumer's risks 
are specified to be equal to or less than a and f3, respectively, 
a feasible region for designing a software reliability 
demonstration test is written as 



(3) 



±WM-i**ft. (4) 

i=0 l - 

In general, the simultaneous equations obtained by 
replacing inequalities in Eqs. (3) and (4) by equalities do not 
always have a unique solution since s is an integer. A practical 
solution would be obtained by using the following three 
conditions: 

(i) The producer's risk is strictly equal to a . 

(ii) The consumer's risk does not exceed j3. 

(iii) Acceptance number s is the minimum. 

IV. Variation Distance Model 

Let Fj and F 2 denote two types of discrete probability 
distribution, and p u and p 2 . (i = 0, 1,2, ...) be probability mass 
functions associated with F 1 and F y respectively. Then the 
variation distance [6] of F 1 and F 2 is defined by 



d v {F 1 ,F 2 ) = Y\p u -p 2i I. 



(5) 



The variation distance in Eq. (5) can be regarded as a measure 
to express the distance between F } and F y In other words, as 
the variation distance increases, we can distinguish F x from 
F 2 more easily. 

This section presents a model based on the variation 
distance for the purpose of determining the values of t and s 
of the continuous model. 

When = 8 , we have 

In the case of 6 = l , we have 

A set of Eqs. (7) and (8) represents a probability 
distribution F , expressed by a probability mass function p 

(i = 0, 1) in Eq. (5) when 9=9 . A set of Eqs. (8) and (9) 
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(6) 
(7) 

(8) 
(9) 



expresses another probability distribution F y characterized 
by a probability mass function p 2 . (i = 0, 1) in Eq. (5) in the 

case of 6=6 l - Hence, the variation distance of these two 
probability distributions is written as : 



d v (F l ,F 2 )=2 



(10) 



We can obtain an optimal pair of values, (t, s)* by 
maximizing d (F , F 2 ) in Eq. (10) since we can distinguish F 
from F 2 more easily as the variation distance increases. 

V. An Optimal Value s* for Each Value of t 

This section shows the existence of the optimal value s* 
for each value of t. 

Let D(s) = d v (F r F 2 ), the difference of D{s) is given by 



D(s+Y)-D(s) = 2 



(t/0 o ) s+1 
( S + l)! 



( S + 1)! 



,-tie, 



(11) 



The sign of D(s + 1) - D(s) follows that 
(i)If 



s< 



^(log^-log^) 

thmD(s+l)-D(s)>0. 
(ii)If 



-1, 



s = - 



1(00-9,) 



-1, 



(12) 



(13) 



1, 



1, 



(14) 



(15) 



000! (log O -l°g 0i ) 
thenD(s+l)-D(s) = 0. 
(iii) If 

s> '(0o -fl) 
000^10800-1080!) 

thmD(s+l)-D(s)<0. 
Therefore, Let 

¥ ^(log^-log^) 

then we have the following theorem. 
Theorem 1 : 

(i) If y/ is not integer, there exists a unique s* that maximizes 
D(s) and s* is the minimum integer which is greater than iff . 

(ii) If y/ is integer, there exist two s* that maximizes D(s) and 
s* are y/ and y/ + 1. 

VI. Relation Between Statistical Model and Variation 
Distance Model 

This section reveals relation between the statistical model 
in Section III and the variation distance model in Section IV. 
d v (F v F 2 ) in Eq. (10) is expressed as following: 

d v (F, , F 2 ) = 2 - 2(Pr[F, 1 O ] + Pr[£i 1 0j) (16) 

using the producer's risk ¥r[E l I0 O ] of Eq. (1) and the 

consumer's risk Pr[Fi I 0J ofEq. (2). 
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Therefore, maximizing the variation distance d(F p F ) is 
equivalent to minimizing the sum of the producer's risk 

Pr^l^] and the consumer's risk PrfFil^J in the 

statistical model. This means to minimize the expected cost in 
case the cost of the producer's risk is equal to the cost of the 
consumer's risk. 

VII. Numerical Examples 

Fig. 1 shows numerical examples of variation distance 
d (F , F ) when acceptance number of software failures s 
varies ins=0, 1,2, 10forf = 1000,2000, 3000,4000, and 

5000 where Q =2000 and 9 l =1000. 

For each value of t = 1 000, 2000, 3000, 4000, and 5000, the 
optimal values are s* = 0, 1,2, 2, and 3, respectively. 
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Fig. 1. Numerical examples of variation distance 



Conclusions 

This study proposed a model of reliability demonstration 
testing for continuous-type software such as that for 
controlling production lines and operating systems of 

computers. When MTBSF on the contract and tolerable 

lower limit for MTBSF { are given, testing time f and 
acceptance number of software failures s are designed by 
maximizing d{F l , F ). This model has less parameters to be 
prespecified than the statistical model. Theorem 1 reveals 
the existence of the optimal value s* for each value of t. 
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